STraTOS 1997 April & May/STraTOS 1 - 1997 April & May.iso/CD01/INTERNET/BROWSERS/PURGE.LZH/PURGE/PURGE.TXT

Text File | 1996-07-28 | 20.1 KB | 587 lines

Purge (NOS/Oasis) V1.2 Doc File
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


Contents:

i) Introduction
ii) History
iii) Installation
iv) Instructions
v) Legal
vi) Contact
vii) Future
viii) Credits


i) Introduction
~~~~~~~~~~~~~~~~

This program was written after a  certain annoying person kept posting
to comp.sys.atari.st.  However I found the  kill file functions, in at
least my version of NOS (0.42) bombed  out when running (2 Bombs).  So
after talking to Rich Tietjens, and asking him, If he'd send me a copy
of a program he'd  written  for  filtering  Fido  Net News,  I started
writing a version for NOS.  I  did  get the source code, after writing
the bulk of the program, and due  to some missing files, never managed
to  get  the  executable  version  to  work.   This  program  probably
resembles Purge, and as such is based,  at least on the idea of, Rich
Tietjens program, called Purge. If Rich hadn't written the program...
maybe I'd never have started this program.  What Rich told me was that
such a program was possible.

The program is known as Purge(NOS/Oasis) so as not to confuse it with
the program written by Rich Tietjens.

See the credits section of this manual for details about the people I
wish to thank.

NB: This version of Purge(NOS/Oasis) has been tested with files
created and used by Oasis V1.35E and NOS V0.42.  The program is
unlikely to work with other versions.... upgrading to these versions
is a good idea if you are using earlier versions.


Typical Usage:

Some uses for Purge(NOS/Oasis) are as follows:

* removal of junk adverts (aka Spam) from newsgroup articles
* removal of 'for sale' articles
* removal of articles by certain people, or on certain subjects
* selecting articles only about a certain subject
* parental control of articles... filtering out things such as obscene
  language


ii) History
~~~~~~~~~~~

Purge(NOS/Oasis) History:


V1.0
----

Original version.... only two  copies  released  to  other people, who
hopefully have tested it a bit !

This version was based on  a  program  called Purge, by Rich Tietjens,
which as I understand was designed  for filtering out messages on Fido
News.

I developed this program mainly while  waiting  for his email with his
program in.  I never managed to run  his program, due to missing files
and the like.

This version probably had quiet a few  bugs (!)  and was really only a
beta test version, designed to see what people initially thought.

V1.1
----

More testing, and a few changes later, about 4 days after the original
program was sent out.  Main changes as follows:

* Shows % of lines scanned during filtering, gives a better indication
of how much is left to do.

* Shows an estimate of time  left,  this fluctuates to start with, but
should settle down.

* Speed increase for  Kill  file,  with  multiple phrases, before took
2:20, now takes 0:38 (should give you  some idea of how much faster it
is !).

* Mouse cursor now changes to busy bee while scanning.

* All temporary files, RSC and  KILL.TXT  are  kept in the same
directory as the main program.

* The program checks to make sure there is now enough diskspace.

* The program now  allocates  all  available  memory for filtering, so
limit of lines and articles is now dependent on available memory.

* possibility of Purge.inf file now  added, to preset number of lines,
articles and kill phrases used, so that  odd variants can be used (eg.
Binary groups, where number of lines is high, and articles low).

* Bug removed... when scanning  oasis  indexed files, the last article
was missed.

* Bug removed...  scanning  more  than  one  file  in  a session meant
problems with file increasing in size, as it was added to another copy
of itself.

* Reminder added  to  re-index  news  if  an  Oasis  news  article was
filtered.

* Allows choice of killed, or passed  articles  to be saved, so it can
be used as a positive or negative filter now.

* Temporary file left on disk,  eg.  killed.txt is now no longer wiped
if the passed articles  are  saved.   This  is  there for checking, or
archival purposes.

* Greater GEM  compliance.   Dialog  should  work  with any resolution
(except ST Low),  background  graphics  area  stored  and  returned on
quiting.  Redundant code removed.


V1.2
----

a few weeks later, after some suggestions and much speed increase:

* Now works between Oasis and NOS, and is called automatically
  (Nb. will still work as V1.1 if run from the desktop).

* Faster than V1.1.  About 17 times faster !!!!

* Due to the above change, disk access was slowing the filtering,
  so temporary files are no longer created, and so less slow down.

* NOS.TTP program added to package, see installation for details

* Installation and other sections added in manual

* Bug removed which was introduced when variable types were changed.
  The bug caused BATCH.TXT articles to not be recognised.

* Bug removed which broke up the headers, path: variable if >256
  characters long.

* SOURCE.INF file allows source file to be saved as SOURCE.TXT for
  what ever purpose you might find it useful :)

* Program now quits after a keypress after filtering news, when run
  between NOS and Oasis.

* News is not saved if 'passed' is selected, and no articles were
  killed.

* Screen no longer cleared, to keep compatibility with Magic

* Bug which meant that first word/phrase in KILL.TXT was ignored fixed


iii) Installation
~~~~~~~~~~~~~~~~~~

Due to the changes that have taken place between V1.1 and V1.2,
I thought it a good idea to add this section

There are now two methods of running Purge(NOS/Oasis):

1) As V1.1, from the desktop:

* Place all the files in a suitable folder, eg. PURGE

* Create/Modify the KILL.TXT file and place it in the
same folder as the other files above.

* Run and follow instructions as below

Nb. KILL.TXT file _must_ be present in the same place as PURGE.PRG and
PURGE.RSC for the program to run.


2) Between NOS and Oasis:

* In the OASIS folder, rename NOS.TTP to NOS.TTX

* copy the following files into the OASIS folder:

PURGE.PRG
PURGE.RSC
KILL.TXT
NOS.TTP

* Do not place the above files in a sub folder, they must exist along
with all the other Oasis files, such as OASIS.PRG and the old NOS.TTP
(now NOS.TTX)


iv) Instructions
~~~~~~~~~~~~~~~~~

Simple, Step-by-Step:

This program is designed only to  work  with Oasis V1.35 and NOS 0.42,
using Oasis Version 2  with  this  program  will  not work.  Using any
other programs files with  this  program  will  almost certainly cause
damage !

Nb. I'm not sure if Oasis V1.3x and NOS V0.4x will work, however upgrading
    is advised, as the latest versions (currently Oasis V1.35 and NOS
    V0.42) contain many bug fixes and functions not present in earlier
    versions.

Switch to a resolution higher than  ST  Low  res, eg. Med Res, or High
Res

Modify the KILL.TXT file to suit your needs (see later)


For Running From The Desktop (as V1.1):

        Load the PURGE.PRG program

        Select the 'PURGE' button

        Select whether you are filtering  BATCH.TXT  file generated by NOS, or
        an indexed *.NEW Oasis news file (Select BATCH for BATCH.TXT
        files generated by NOS, or Oasis for *.NEW files generated by
        Oasis).

        Find the correct file using the fileselector

        Wait while Purge Filters out the articles

        Select whether you want to save the killed articles (those which _did_
        contain one or more of the phrases  or words in the KILL.TXT file), or
        the passed articles (those which _didn't_  containing one or more of the
        phrases or words in the KILL.TXT files).

        Quit the Purge program and re-index/index the filtered news file.


For Running Between NOS and Oasis:

        Run Oasis as usual, select the internet option, and connect using the
        correct option.  You should see as the screen clears a small line at
        the top of the screen, telling you that the program is running.

        Run NOS as usual

        Exit from NOS (by typing 'exit' at the NOS> option

        If there is any news to filter Purge(NOS/Oasis) will now run,
        select the PURGE button if you want the program to filter the
        news, otherwise select Quit.

        Instead of asking the user to select the file to be filtered,
        Purge(NOS/Oasis) will automatically load and filter the file
        called batch.txt in \OASIS\SPOOL\ARTICLES which is where NOS
        saves the news

        Select which articles you want saving, the Passed or Failed
        articles.

        Quit from Purge(NOS/Oasis) by pressing a key.

        Oasis will now run, and should either automatically unbatch
        the news (if you have this set (*recommended*)) or unbatch on
        your command.


Nb. for ease of use for the average user I suggest you run
Purge(NOS/Oasis) between Oasis and NOS, this then doesn't require the
user to think about filtering the news, and should also filter quickly
enough for it not to be too much of a disturbance.


Detailed:

The first thing you need to do is  to edit the KILL.TXT file in a word
processor.

This is  an  ascii  file  which  contains  the  words  or  phrases you
want/don't want to find in your news articles.

And example is shown below:

Sinclair
Subject: FS
Subject: WTB
Subject:FS
Subject:WTB

Make sure there is a CR after the last word/phrase. Otherwise the
program  will miss the last phrase/word. DO NOT put an extra CR on a
blank line... otherwise _all_ articles with a blank line in will be
killed !

The KILL.TXT file needs each new word or phrase on a new line.

Beware of putting in something like

WWW

as many people include things like:

http://www.mypage.com/myhomepage/home.html

in their taglines, and the www will  be picked up.  Instead add spaces
like:

 WWW

so that only complete words where WWW appears (like that) with a space
either side.

beware also of things like:

CAB

which unless spaces are used as above, will apear in words, such as:

Cable
cabin
cabbage

Again use of spaces can limit this to words.

Also beware of using things like FS  alone, use as in the example, the
subject line as well.

Before you begin using the  KILL.TXT  file  its  a  good idea to check
through your news file in a Word  Processor, first for the phrases you
have specified, and to check that they only come up where they should.

Also you might like to consider only working on a backup file to start
with.



Purge (NOS/Oasis) can work with two  sorts  of source file, that saved
by NOS,  the  BATCH.TXT  file,  often  found  in  \spool\articles\  or
oasis\spool\articles.

This contains  _all_  the  news  articles  saved  out  by  NOS  as  it
downloaded  them.   If  you  have   Oasis   set  to  unbatch  articles
automatically after quiting NOS, then you  are only likely to see this
file if you have a crash.  Oasis  can be set to _not_ unbatch articles
on quiting NOS, but only when you specify, using the unbatch option in
the Oasis options file.  Once you  want  to unbatch articles, from the
BATCH.TXT file (filtered or not)  use  the Un-Batch News option, under
the Internet menu in Oasis.

The above change is not needed if you intend to run Purge
automatically between Oasis and NOS, indeed letting Oasis unbatch the
news straight away is a much better option.

If you already unbatch or index the news with Oasis, you can still use
this program, provided you  are  using  the  correct  version of Oasis
(1.35 has been tested, other version  are untested).  Oasis stores its
news  files  in  /oasis/spool/news/newsbase/   as   *.NEW  files,  eg.
96203001.NEW, they are usually the large  files in the folder.  If you
filter these file, you must   remember   to   use  Oasis to 're-index'
the group(s) before reading them   in  Oasis,  otherwise the displayed
text will be totally out  of  place.    Use  the Re-index option under
the news menu.


Purge (NOS/Oasis) doesn't overwrite the source file until all articles
have been sorted, and as such is  unlikely  to  corrupt  the source
file, during errors, unless the  error  occurs  while  saving  the
articles (after filtering).

Purge (NOS/Oasis) automatically allocates memory  for the source file,
and as such provides a limit as to how big the source file can be.  In
the 3 Meg of memory I have left, it allocates space for 33000 lines of
text, 5000 articles and about 2000 Kill words and phrases.

The number of lines assumes all lines to be 80 chars wide, the average
from comp.sys.atari.st varies, but is about 40.

If you want to alter these values... for instance if you are filtering
a binaries group, where there  are  very  low numbers of articles, but
very high numbers of lines.  Or  a  contact group, such that there are
many short articles then you  can  alter  the values using a purge.inf
file.

The file is checked  on  loading,  and  used.   If  there isn't enough
memory for the values you have suggested, then the program should tell
you this, and quit.

The file is also plain ascii:

Purge (NOS/Oasis) Config File
30000
5000
2000

the first line must be exactly like this, and is a header, which Purge
checks to make sure it is using the correct file.

The next line is  the maximum number  of  lines  of  text  that can be
loaded.

The next value is the maximum number of articles that can be loaded.

And the last number is the  maximum  number of kill words/phrases that
can be used.

The above is what I would use for most groups (like comp.sys.atari.st)
however for a binaries group I may use:

Purge (NOS/Oasis) Config File
50000
500
200

and for a group with many short articles:

Purge (NOS/Oasis) Config File
15000
20000
200


However use of this file, is  only  really  needed if the setting used
normally don't provide enough of  one  type of variable.  Don't fiddle
with this unless you normally get errors, and need to !



Creating a file called SOURCE.INF  in  the same directory as PURGE.PRG
and PURGE.RSC will  cause  the  program  to  save  the  source file as
SOURCE.TXT in this same directory as Purge(NOS/Oasis), before  it
writes over the original with the newly  filtered  text.   The  file
can  be  created with any program, and contain anything, its presence
is what causes the program to work.



v) Legal
~~~~~~~~~~

Purge(NOS/Oasis) executables, document file and other related files
may be distributed together freely in an unmodified form and may be
used without fee by any individual for non-commercial use, subject to
the following terms:

Purge(NOS/Oasis) may not be sold or resold, distributed as a part of
any commercial package, used in support of a commercial service, or
used in any kind of profit-generating activity even if it is being
distributed freely.

The integrity of the original Purge(NOS/Oasis) archive file as
distributed by me is essential.  None of it's contents may be
modified or removed or added to in any way.

Purge(NOS/Oasis) is provided "AS IS" without warranty of any kind,
either express or implied, including but not limited to the implied
warranties of merchantability and fitness for a particular purpose. In
no event shall Philip Hough be liable for any damages whatsoever
including direct, indirect, incidental, consequential, loss of
business profits or special damages, even if Philip Hough has
been advised of the possibility of such damages.

If the user does not agree to  any  of  the above then they should not
use, modify or distribute the program or its files.


vi) Contact
~~~~~~~~~~~

You can email me at:

[e-mail]

or by snail mail:

Phil Hough
23 Pen-y-bryn Way,
Newport,
Shropshire,
ENGLAND.
TF10 7ER

Chances are if you use this program, you have/use NOS 0.42, and Oasis
1.35E, and so you have access to email.  As this is the case its
probably not difficult to post me an email telling me what you think
of this program.

I know this program is freeware, and there is no obligation
what-so-ever to pay me any money, send me any stuff, or even to bother
telling me you use the program.

However the last of these would give me an idea of whether people are
actually using my program.... one of the sole reasons why I wrote and
released it (lets face it, if I wanted the money I'd make it
shareware).  I'm not bothered about knowing your address and phone
number so I can hassle you for money latter.... for all I care send me
an anonymous email.  I'd really just like to know people are using it.

The more people that contact me, and tell me that they use it the more
likely that I'll write and release other software in the future (take
that as a threat if you like <g>), both on this platform, and any
others I end up using in the future.

If nobody bothers to contact me... Its likely that in the future when
I'm writing something for myself (which is how this program started
out) I probably won't think to add a user interface and take the
trouble to test it, and then release it (believe me this program has
been tested... read the history section for the changes and bugs
found).  All in all, I'll end up not releasing software for others to
use, and keep it to myself, simply because I won't know that people
want/use my software.

If you'd like the GFA Basic 3.6TTe source code for the program then
also contact me and you can have it, with  a few provisions :)


vii) Future
~~~~~~~~~
The program will continue development so  long as there are changes to
be made, or problems to sort.   If  you provide me with feedback, then
the program is  likely  to  continue  development.   If  I receive no
feedback at all then the program  will most likely stop development at
V1.2.  To provide feedback just send  me  an (e)mail saying if you use
the program, and if  there  is  anything  you'd  like  to see added or
changed.

Some things I'm thinking of at the moment for V1.3, are the removal of
the estimated time gizmo: on a 550K, 330article news batch it takes so
little time to filter now that loading and saving take longer, and by
the time the estimated time gizmo has settled down, its 80% of the
way through the filtering, and the updating of the dialog to tell you
how far its done, now adds significantly to the time taken to filter.

I may also think about re-introducing the feature (ie. an artifact of
temporary files) that saved out the killed articles, as well as those
that had passed (ie. save _both_ out, just to different places).

Let me know what you think on the above.... I'm unsure if either
change is really needed, or whether people want or don't want either
of the above.

For _my_ purposes the program as it stands, its in my eyes, as close
to my ideal as it is going to get, its recent speed increase, and the
functions it has, are all I need.  As such if noone tells me
differently there is little point in changing or adding to it... so
let me know !


viii) Credits
~~~~~~~~~~~~~

I thought it about time to add a section in this document to express
my gratitude to the following people, without their contributions this
program would not have been written, nor have developed.

Rich Tietjens:  For telling me that a program such as Purge(NOS/Oasis)
could be written, and that at least one other person thought this sort
of program could be written.  If Rich hadn't contacted me, I'd
probably still be asking around the newsgroups, asking if such a
program exists.

Nick Flintham: for testing the software, and providing valuable
feedback and ideas.  For anyone interested the following were
suggested by Nick:

* Option to Select Kill or Passed Articles
* The option to place Purge(NOS/Oasis) between NOS and Oasis so that
it can be called automatically.

Phil Yeadon: for a great deal of help with the programming of
Purge(NOS/Oasis), including teaching me how to program GEM'd Dialog
Boxes, and also how to use memory reserving to run Between NOS and
Oasis.  I also want to thank Phil at this point for writing Oasis,
which without, I may not have got this far.

To the people above.... many many thanks